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This  paper  considers  the  problee  of  obtaining  an  optlMl  control  law,  which  is 
constrained  to  be  a  linear  feedback  of  the  available  aeasureaents ,  for  both 
contlntMus  and  discrete  tine  linear  syoteas  sub^scted  to  additive  white  process 
noise  and  aeasuregent  noise.  Necessary  conditions  are  obtained  for  ainlaising  a 
quadratic  perforaance  function  for  both  finite  and  ioflntte  teralnal  tiae  cases. 
The  feedbaoh  gains  are  constrained  to  be  tlae  Invariant  for  the  Infinite  terainsi 
tisM  eases.  For  all  the  cases  considered,  algorlthas  are  derived  for  generating 
sequences  of  feedfback  gain  aatrlces  which  successively  l^rove  the  perforaance 
fUMtion.  A  continuous  tlae  maerlcal  etsaple  Is  Included  for  the  purpose  of 
deaonstratlon. 
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OPTZNAL  OUnVT  FODBACK  COVTROL  OP  LUSAP 

tnmmm  n  piusbse  op  porcdic  ae)  keasurdovt  ioise 
Be  SurMb  M.  Joshi 

aiMiAity 

tte  froBXfli  of  olitolaiai  *b  optiaol  eoetrol  low,  vbich  it  ccwatrtLined  to 
bo  ft  fftofbfttb  of  tbo  ftvftllftblo  ooftottrooftott,  it  eooaidtrod  for  both  continuous 
ftad  41aey«%o«tlat  liator  tytt—  aobjtetod  to  adAltlTo  vhitt  procott  nols*  tod 
MftOWOMftit  aolM.  Bftooft 0017  eoodltioM  oro  obtolaod  for  ■Inioltlnc  o  quadra^ 
tio  pMTfonMMOO  fboetloo  for  both  fialto  and  laflAito  torolnal  tiJM  cftoot.  Tbo 
foodbftok  fftift  aotriooft  aro  enootrolftod  to  bo  eoootaat  for  tbo  laflaito  torolnal 
tlao  oftooo.  For  all  tbo  eaooo  eoooldorod,  alcoritbno  aro  dorlvod  for  •ooorat> 
iof  aoqpMMOOO  of  foodbaok  foia  oatrleoo  obieb  ooecoooiroly  iaproro  tbo  porfora- 
oooo  fbootlco.  A  aoBorlooI  on^plo  la  clvoa  for  tbo  purpooo  of  dMoootratlon. 


Ooaterol  bpotOB  iooi4pi  for  procootoo  obieb  ooffor  frao  proeooo  and  aoaoaro> 
atoA  ondoo  to  m  ibpnn  tat  probloa.  It  tbo  proeooo  dpwiieo  aro  linoar,  and  if 
tbo  irooooo  aoloo  and  tbo  ooaaoroaMMat  aoioo  aro  Oanooian  and  obito,  it  it  voU 
boBWO  tboA  tbo  oooArol  loo  obieb  alntaiaoo  a  qoadratie  pcrforoaneo  fonetloo  It 


a  iiboor  fbMtiob  of  tbo  optlaal  ootianto  of  tbo  otato.  Tboa,  tbo  oppIleatAoo 
tbia  oooirol  Im  aaoooaitadoo  tbo  uao  of  an  oollao  atoto  ootlaator.  This 
iootio  agprootb  aiqr  not  bo  attraotim  fran  a  proetieal  rloopoint  bocauao  of  tbo 
Mgb  eoot  of  tbo  additional  oquif^  roquirod.  la  addition,  tboro  csitta  a 
dangarono  poMibdlitp  of  dirorgoaeo  of  tbo  atoto  ootlaator  oboe  tbo  ooioc  Input 
aatrlt,  aad/or  tbo  aoona  of  tbo  forcing  and  nooauraaoot  nolte  procoaoot  aro  not 


inn 


iBKJWB  •eeurfttvly. 

For  this  rsMoa,  th«  dtsign  of  a  satisfactory  controller,  vhlch  needs  the 
feedback  only  of  the  available  plant  neasuresMnts ,  is  a  problem  of  imense 
practical  iiqiortaace.  There  have  been  ntaerous  recent  attempts  in  this  area. 
These  have  proceeded  mainly  in  two  directirais:  pole  shifting  techniques  using 
output  feedback t  and  minimisation  of  a  quadratic  performance  function  using 
output  feedback.  Ihe  former  procedure  is  necessarily  confined  to  detersdnlstic 
systesMt  vhile  the  latter  procedure  is  also  useful  for  the  practically  impor¬ 
tant  stochastic  ease,  in  the  sense  that  the  apriori  knowledge  of  the  noise 
statistics  can  be  used  to  advantage.  This  paper  considers  the  latter  approach. 

The  nois^free  version  of  the  linear  quadratic  optimal  output  feedback 
e<»trol  prohlms  mas  considered  in  references  (1)  through  (k).  The  basic  philo- 
soplqr  was  to  minimise  the  performance  degradation  caused  by  the  constraint  on 
the  control  lav.  The  coostrainsd  optimal  control  law  depends  on  the  initial 
statei  therefore,  the  approach  was  to  minimise  the  value  of  the  performance 
fttnetioo  averaged  over  the  initial  state  (with  apriori  known  statistics), 
(refaraneee  (3)  and  (b));  or  to  alnlmisa  tha  "worst”  value  of  the  performance 
fumetiom.  wham  the  initial  state  is  known  to  lie  within  a  hyparalllpaold  in  the 
ststte  spane  (raf.  (2))i  or  to  minimisa  the  mast, mam  ratio  of  suhoptlmal  and  op¬ 
timal  vmluea  of  tha  parforaamea  fhoetioms  (rtf.  (1)).  Zn  all  these  csees.  the 
prehlsm  wae  fimally  reduced  to  a  eoaplas  aomllaaar  optiadsatioo  problsm. 

Xn  refarsmsaa  (3)  aad  (b),  the  noiae-free  cae#  wae  eomsidered,  and  the 
aigahraie  naetasary  ecmditlome  ware  derived.  Zn  additioa.  deaign  of  optimal 
dymemle  eeapsaamtcrs  of  a  praepacifiad  order  was  also  discussad  la  refarm^e 
(b).  In  rafsrsasa  (3)«  Assitar  eomsidared  tha  stochastic  prohlsm,  with  white 
pameaas  seisa.  hmt  no  msasmrmisnt  aolsa.  aad  darivad  tha  naceaaary  eooditioas 
for  optiamlity.  NeXiaaa  (ref.  (d)},  ccmaigered  the  above  problem,  with  etate 
•md  eoatrol  depaadmnt  forcing  aoisa,  hut  no  msaswamt  nolea.  All  the  efforts 
deaerihed  above  ecmaidared  omly  eoatiamoma  time  syetaas.  Zn  addition,  measure- 
mmti  sedsa  was  assmms<  to  ba  abeeet.  Zm  practice,  bowever,  swaeuraaent  noise 
is  slaoct  always  prssamt ;  tharsfore,  it  is  important  to  eonsitor  the  degrading 
effact  of  the  direct  tremcnlttal  of  measursment  noise  through  output  feedback. 

Zs  ref srsmca  (7).  the  dieerete-tiae.  finite  terminal  time  problem  was  considered 
amd  saeasmary  ccmdltioma  vere  obtained  using  dynamic  progremlng.  Bowever.  the 
imflmite  tarmlmal  time  problem  wae  not  considered,  aad  a  minimising  algorithm 


WM  not  doTolopod. 

Tho  purpoM  of  this  paper  Is  to  consider  both  continiious-tlBe  and  dlscrete- 
tlae  linear  systsas  i^ch  are  subjected  to  both  forcing  and  measurenent  noise. 

The  necessary  condltl<MBS  for  optlnallty  are  derived  for  continuous-time  systems 
In  the  first  part,  using  the  matrix  minimum  principle  (ref.  (8)).  Both  finite 
and  Infinite  terminal  time  cases  are  considered,  the  control  lav  for  the  latter 
case  being  emistralaed  to  be  a  constant  feedback  of  the  noisy  measurements. 
Algorithms  are  derived  to  give  sequences  of  successively  better  control  gains, 
tar  both  finite  and  infinite  terminal  time  cases.  The  discrete-time  case  Is  next 
CMMldered.  The  necessary  conditions  are  derived,  using  the  discrete  matrix 
minimum  principle,  for  both  finite  and  Infinite  temiaal  time  cases,  for  the 
finite  terminal  time  ease.  It  is  shovn  that  the  necessary  coi^ltlons  coincide 
with  those  derived  in  reference  (7)  using  dynamic  programing,  while  for  the  in¬ 
finite  terminal  tins  case,  the  feedback  galas  are  constrained  to  be  ccmstant. 
Algorlthme  are  developed  for  all  eases  to  generate  sequences  of  feedback  gains 
which  smceessivmly  improve  the  performance  function.  A  ccmtinuous-time  numeri¬ 
cal  sauniple  la  give  for  the  purpose  of  demons  tret  imi. 
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n  s  a  syetem  matrix 

asm  lapot  matrix 

t  X  a  output  matrix 

empected  value  oporator 

m  X  t  feedback  matrix 

m  X  t  matrix  deflaed  la  the  text 

■wlltoalaa 

lagraagiaa 

ladicea 

perfaraamce  fuactloae  for  tbe  flail*  and  laflalte 

termimal-tlme  caaee 

n  X  a  coetate  matrix 
a  X  a  lagraage  amltipllcr  matrix 
tcrmiael  time  for  tba  discrete  ease 
a  B  a  Rlecati-type  matrix 
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n  X  n  velghtiog  matrix  for  the  atate  vector 
a  X  m  veli^ting  aatrix  for  the  input  vector 
n  X  n  tenilnal  state  weighting  aatrix 
Trace  operator 
time 

terminal  time 
mr*vector  input 

a  X  n  forcing  nolae  eovariai^e  matrix 

n>Tector  forcing  noise 

t  X  i  measurement  noise  covariance  matrix 

t-vector  measurement  noise 

ii>4imemsional  state  vector 

i-dimemalooal  output  vector 

Dirac  4elta  fttaetioe 

IroeecluMr  delta  fuaction 

tt  X  a  state  covariance  matrix 

etee4y*atate  value  of  t 

tiM  tremapoee  of  a  aatrix.  mad  "-I*  denotes  the 


ftmlte  thmlmal  Time  CSae 
Ike  ifatam  la  glvam  Iqt 

•  A(i>x(t)  ♦  •(t)»(t)  ♦  v(t)  (1) 

fix}*  C(t)x(t)  ♦  v{t)  (2) 

aiimra  A(%},  Kt).  C(t)  are  a  x  m.  a  x  m.  sad  I  x  a  aatrleea  and  s(t},  uCt).  amd 
f(t)  are  ramjmatJvaJjr  a  x  1.  m  x  1.  amd  I  s  1  stale  vector,  input  vector,  sad 
amtpM  vmetar.  v{t>  and  Wt)  are  ehite  noise  proeeeaea,  vlth  taro  means,  and 

C(v(tW^(T)|  •  T{t)d<t-t)  (3) 


I 


E[w(t)v^(T))  »  W(t)6{t-T)  (1 

vhAT*  V(t)  >  0,  W(t>  >  0  are  n  x  n  and  I  x  1  natricea,  and  6(*)  la  the  Dirac 


dalta  fUBCtioB.  Also, 


Ctv(t)w  (t)1  »  0 


Tha  initial  stata  covariance  natrix  la  aaauaed  to  be  hnovn: 


etx(o)x^(o)l  •  I(o)  -  I 


Conaidar  tha  problaa  of  ainlaialBC  the  functional 


I  c[x'^(t^)8x(t^)j  ♦  i  [x’(t)Q(t) 


x{t)  ♦ 


u(t)j 


wtonra  Q(t}  >  0»  R<t)  >  0,  8  >  0  are  e  x  a.  a  x  a,  and  o  x  n  aatricea.  &ippoae 
thm  control  lav  ia  raatrictad  to  ba  a  feadbach  of  tba  output. 

tt(t)  •  0(i>y{t) 

•  0(t)C(ihiU)  • 


alMra  0(t)  ia  tba  nil  oatfait  faadbacb  natrix. 

tbia  fonaalatioa  allewa  far  direct  traanaittal  of  ebita  aolae  Into  tba 
i^«i  via  faedbaab.  It  la  claarly  aaea  tbat  tba  tera  Cttt^(t  )i(t>tt(t)}  ia 
iafiaita,  alaaa  fC»<t)w^(i)I  •  »f(t)8(t-f).  TWa  can  ba  avoldad  by  iacludlaf 
only  tbat  part  9t  tba  control  ananCf  vbicb  doae  aot  contain  vbita  aolaa.  Ttaua 
tba  toXlontat  nadlfiad  parfomaara  fuacttoa  Ic  eonaldarads 


J  •  |fla''(tj)«x(t^)) 


^  iJ^'^x^CtWtla(t)  •  <0{tH:tt>x(t))%(t>(0(tK(t)xtt)|«t  (6) 


tba  m»mmrmm%  aolaa  ta  ant  diraatly  Ineladad  ta  tba  parfomaata  function, 
bat  tba  dngrndimi  affwt  baenana  of  ita  faadbacb  ta  Indeed  tabce  into  account 
via  tba  atata  eevariaaaa  evolatloa  etiaatloa.  Tbaa,  alaca  tba  etate  covariance 
natrix  dapanda  on  tba  product  of  tba  feedbacb  fain  antrlx  and  the  naaaurenaat 
aolaa,  tba  aaaawnaaat  noiee  dapendewt  part  of  tba  input  elgnal  vlll  be  auto- 
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■ftticAUy  P*nali—<i  Id  iproportlon  to  Its  eovarlsnce  aatrix  V,  ss  vlll  be  seen 

Tliis  fcHnaulDtion  tlms  provides  s  design  tecbnlque  in  vhlch  the  knowledge 
of  tbs  noise  stetlstlcs  esn  be  used  to  sdvsDtsge. 

Tl»  evolutioa  of  the  covsrlsBce  DStrix  of  the  stete  x{t)  cen  be  easily  de¬ 
rived  ss 


•  U  ♦  ICC)l  ♦  I(A  ♦  BOC)^  ♦  BCWcV  ♦  V  (9) 

m 

where  Z(t)  •  CCx(t)x  (t)J,  I(o)  •  I  sad  the  syabol  ‘(t)  is  dropped  for  con- 

o 

vsAieaee. 

The  perfnrasMie  fuaetioo  of  equ*tioB  (8)  eea  also  be  written  as 

J  •  |Tr(«I(t^n 


« 


1 

2 


f' 


Trl(Q  ♦  cV*SC)tl  dt 


(10) 


Xt  is  rsBpIrsd  to  Bii^Blse  the  i  la  (10)  with  reepeet  to  0  subject  to  the 

eoMPtmsft  (9). 

Xt  is  sew  pMSihle  to  apply  the  aotrls  •iaisMS  prtoetpie  (ref.  (8)). 
Deftae  the  twtlteaisa 


■(t.0,1)  •  ^  «  C^0%0C)t|| 

•  Tr  •  IDC)t 

•  ♦  t  |l*j 


•  XtA  ♦  IDC)' 


(U) 


where  Kt)  Is  se  s  s  s  eartrU  of  eeetete  verlshlee.  the  LeltUl  stete  eo- 
verieaee  SCO)  •  aed  the  aetrisee  V(t)  ael  tf(t)  are  siws<  to  he  feeswe 
sprieri.  Osieg  the  eetris  etaJUMs  priesliae,  aed  the  predieet  astrleee  de¬ 
rived  ta  rsfhrsess  (•},  the  sesssssry  eeadlUoae  f»r  optieallty  em  he  derived 


WCtC^  •  •  i^rtov  •  0 


•  iBC)*f  •  ma  •  ioc)  •  (a  *  cVmxn] 


(12) 


(13) 


dt 

»  (A  ♦  BGC)!  ♦  I(A  ♦  800"^  ♦  BGWg'^b'^  ♦  V 

(lU) 

£(0)  » 

(15) 

P(t^)  »  3 

(16) 

In  tiMM  cquAtlons,  P(t)  •  2K(t)»  and  the  symetry  of  P(t)  and  K(t )  is  obtained 
duriac  th*  derivation  becauae  of  the  sywetry  of  the  right-hand  side  of  (13), 
and  beewjae  8*8.  Equatlona  (12)  to  (l6)  describe  a  nonlinear  two-point 
boundary  value  problea.  for  a  given  ‘3(tJ,  equations  <13)  and  (li>)  are  linear 
in  P  £,  while  for  given  P  an#i  equation  (12)  is  linear  in  u.  It  is  inter- 
eatlttg  to  note  that,  although  the  aeasureaent  noise-dependent  portion  of  the 
control  signal  was  not  weighted  in  the  perforaanoe  function,  the  neasuresMent 
covariance  aatrtx  V  does  tend  to  reduce  the  feedbaeh  gain  0  in  equation  (12). 
as  expected,  for  the  case  with  perfect  neasurenents  (W  ■  O),  the  necessary 
conditions  reduce  to  those  of  Assiiter  (ref.  C^)).  while  for  the  noise-free  case 
(V  •  0,  V  •  0),  the  necessary  conditions  reduce  to  those  due  to  Levine,  Johnson, 
and  Athnns  (ref.  (h)). 


infinite  Duration  Case 

The  secessary  CMdlttone  obtained  shove  require  the  solution  of  a  conples, 
tan-polst  boundary  value  prohlea,  and  the  fsedhach  gals  0(t)  obtained  Is  ttsw- 
varyisg.  A  satwral  estenslon  of  this  probtsn  is  the  cons  teat -coeffl  cl  set, 
iaftatte  teevtsal  tine  ease,  etth  0  cooetralned  to  be  a  ttae- levari  ant  aatrls. 
(Jhdar  these  ctrcuMrtances,  with  A.  1.  C,  b  constant,  for  a  stabliuit^  out¬ 
put  fsedhach  gain  astrls  it  It  well  hncen  that  the  cowarlancs  swtris  £{t) 
tends  to  s  eosstssrt  aatrls  T  >  0  ss  t  *  «•.  In  the  treataent  below.  It  Is  as- 
sunsd  that,  for  the  V,  (1,  V,  and  the  dyienlcs  under  consideration,  f  aalsts  and 
is  s  pneitiwa  deftaite  satris.  This  wenild  indeed  be  true  If  (f  •  B  )  is 

pesitive  deftaite. 

Sa  this  ease,  if  the  terainel  tiae  t^  «  «,  the  perroraance  functson  of  (lO) 
util  be  taftslte,  stsce  the  Urtegrand  tends  to  a  eoe&sta«t  value.  Therefore,  it 
is  assMdsgfvii  to  ^aiaiie  the  ^eoet  rate* 

1  *  it*  ^  Tf|(^  •  cVhocil}  it  (IT) 

t^-*^ 
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or  J  -  i  Tr|(Q  ♦  cVrGC)SJ  (18) 

vlMr*  {  i»  thm  ttMudOr  state  (positive  definite  by  assuaption)  solution  of  the 
e<ivariaBee  equiatioe  (9)>  fbais,  the  prohlea  reduces  to  a  static  optiaitation 
pr^daa.  Defialac  the  lacrancian 

id.O.K)  •  I  Tr|(Q  ♦  c’g^RGOI] 

♦  Tr  ♦  aoc)!  ♦  t(A  ♦  BGC)^ 

♦  acycV  ♦  (19) 

where  t  is  a  a  a  a  eoaataat  aatrix  of  Lacrai^ie  Multipliers >  the  aecess&ry  con> 
dltioM  for  a  aiaif  are 

^•0,  ^*0  (20) 

ar  ^  it 

eldeh  fiaally  reiaee  to  the  steedy^'etate  fores  of  equatioas  (12),  (13),  and  (Iti), 
wttli  t  refdeoei  Iqr  t,  ssmI  P  •  2l  •  P^.  Thus  the  aeeeeaary  eoaditloea  for  the 
iaiPiaAte  tentlahl  ties  prohlea,  with  eoatrol  •alas  eoesiraiaed  to  he  eoaataat, 
ere  slisltaaeone  aeallaear  setria  al«ehrele  eqaatloas.  It  hbotild  be  noted  that 
the  aedao-free  ease  is  eat  a  siaiple  eateasloe  of  these  results  since,  la  that 
ease,  f  •  0  far  a  atehle  0. 

A  BtewirUel  Alaorltha 

CaaaHeriat  ftrert  the  ftaite  termlaal  tljw  case,  let  0^(t)  aatf  G^(t)  be  two 

fsedbesh  false,  sash  that 

^  •  U  ♦  »*c)r^  •  r*(A  •  10*0^  ♦  •  t  (pi) 

i  •  0,  i 

^  •  -  ju  •  »®c)V  ♦  p®(A  •  10^0  •  Q  •  c^o®*i«®c]  (») 

After  a  laagthp  elfhhrele  aaalfwletliMi  as  oetlteei  la  Afppeadia  A,  it  can 
be  prevef  that 


a 


J(0®)  -  J(G^)  • 

|rG°CE^c'^  ♦  b'^P^E^c'^  ♦  b'^P^BG^wI 

-  jiic^cE^c'^  ♦  B*^p®r^c^  ♦  b'^p^bg^I^r”^ 

IrG^CE^c'*’  ♦  b'^P^E^c’^  ♦  B’^P°BG^jj(CE^c'^)"^  dt 

♦  I  Tr  f  ^  [{G®-  G^  )‘^b'^p‘^B(G°-  G^)jw  dt  (23) 

Jo 

Ttus,  If  G^  is  eliot«a  to  antlsfy 

Bc^cr^c*^  ♦  B^P°E^C^  ♦  b'^p^bgH/  •  O  (2U) 

«•  h«v« 

J(C°)  -  J(C^)  >  0  (25) 

Tliua,  ft  ■ialftltiai  aJiioritlai  !• 

(ft)  ehooftft  fta  Iftltlftl  G^(t) 

(^)  obtftda  P^(t)  uftiac  {22) 

(e)  ftolvft  (2%)  tad  (21)  ftlaultftnftouftly  for  E^  ftad  G^ 

(4)  go  to  (^)  ftJTtor  tnerftftftlac  th«  ftupftrftcrlpt  tor  uolty 
Tfemft,  ft  ftoecoftfttvft  raductioo  lo  J  ta  oibtmiomd.  lo  the  proof  of  the  algo- 
rttlai,  tt  bftft  W«i  ftftftUftod  that  £(t)  >  0.  This  vill  he  true  If  E(0)  >  0; 
howftVftr,  it  ift  not  aoceeftory  that  £(0)  be  poeltive  definite  for  £(t}  to  be 
poftltlee  definite. 

fear  the  infinite  ternianl  tine  cft«e,  efter  n  elnllnr  ■nnlpulfttlon  (glren 
in  tlM  i^fTpeadin  A),  it  can  be  eboim  that  equatioo  (23)  bolde  eith  the  Integral 
•ilpw  rMweed  and  repdaeinc  TbaHrefore,  the  ftl4prlthB  given  in  etepe 

(ft)  «  (d)  vill  fttill  hold,  vlth  equatione  (21)  aod  (22)  replaced  bp  their 
nteadi^fttate  vearaiona,  if  each  G  le  ttable,  t  >  0  at  each  ctege. 
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A  NUMERICAL  EXAMPLE 


The  follovlng  continuous-time  system  is  considered  for  the  piu  p. 
demonstration  of  the  technique  developed; 


The  perforaance  function  to  be  minimized  was  that  in  equation  (17),  with 


Tba  developed  in  the  text  vae  used.  At  each  iteration,  the  simultau- 
eoue  noalineei*  equmtione  in  T  and  G  (equation  (12)  and  the  steady-st^at o  version 
of  expiation  (Ik))  were  eolved  using  Newton’s  method.  The  initial  feedback  gain 

matrix  vae: 


-9.9 


-6.7 


Th«  ebole«  vm  arbitrary*  the  only  reetrictlon  being  the  stability  of  the  matrix 
(A  ♦  BO°C). 

The  jtrograa  converged  after  five  iterations*  to: 

[-6.879  E-01  -5.196  E-01 

-9.861  B-01  -3.993  E-01 

At  each  iteration*  it  vaa  verified  that  (A  *  BGC)  was  stable.  Positive 
definiteness  of  Z  followed  because  of  the  positive  definiteness  of  V.  The  value 
of  the  perforaanee  function  for  the  initial  G  was  3.i»66*  which  was  reduced  to 
0.3^U3  at  the  final  Iteration. 

DISCRETE-TIME  SYSTBC 

finite  Terminal  Time  Case 
Consider  the  discrete-time  system 

x(kn)  *  A(k)x(k)  ♦  B{k)u(k>  ♦  v(k)  (26) 

y(k)  •  C(k)x(k)  ♦  w(k)  (27) 

lotatioiia  and  dimensions  are  the  same  as  In  the  continuous  case.  v(k)  and  v(k) 
are  sero  mean*  white  noise  processes*  with 

EIv(k)v^{l)J  •  V(k)«(k-l)  (20) 

Clw(k)w''(i)J  -  W(k)«(k-l)  (29) 

and 

€(v(k)w''(l)J  •  0  (30) 

E(x(o)s^(o)]  •  E(o)  •  (given) 

o 

Where  d(*)  is  the  Ibwoeeher  delta  function. 

?(k)  >  0,  W<k)  >  0 

The  perfoi— ace  fuactioa  to  he  ainiaitsd  is 


U 


J  ^  c 


h 


(k)Q(k)x(k)  ♦  u^(k)R(k)u(k)|  ♦  |  E[x^(*)Sx(M) ] 


mkjaet  to  (26),  (27),  ood  tbo  rootrletloo  that 

u(k)  •  G(k)y(k) 


G(k)C(k)x(k)  ♦  G(k)v(k) 


Q(k)  >  0,  *(k)  >  0,  8  >  0 


fiM  KroMOkor  4*lto  fuaetloo  it  ■athooktlcaLlly  v«ll  dofinod;  thus  the  performance 
fttmetloB  of  ovtatios  (31)  eaa  be  ei^plified  aa  given  below: 

J  .  I  V  Hr  ^|Q(k)  ♦  C^(k)o’{k)»(k)C(k)C(k)|  E(k)j 


♦  J  r  tr(0^(k)R(k)0(k)ii(k)) 


♦  |Hr{tf(i)| 


Xt  le 


•eeevtiaf  to 


eiov  tbat  tke 


■otrls  £(k)  ■  E(k(k)s^(k)]  evolve* 


S(fevi)  •  (A  e  «IC)£(k)(A  ♦  lOC)^  ♦  BOMoV  ♦  V 


Is  entiai 


i»  (33) 


Itiai  (3^)t  the  ayHbol  •(k)  baa  beam  dropped  for  eoovenleoee  la  moat  of  the 
^haad  aide  variable*.  Ttaaa,  tbe  problia  reduoaa  to  tbe  alalaitatioo  of  J 
9)  aidkiaet  to  tbe  eonatralat  (3b),  with  £(0),  V(k)  and  ?{k)  known  aprlori. 
diaeroto  aBtrln  alaiaMi  prlaelple**  given  la  reference  (6)  can  be  readily 


l(SU).0(k),K(kvl)) 


|(^(k)  ♦  C*(k)o’^(k)l(k)0(k>C(k))r(k)|  ♦  |Tr  |o^{k)R(k)0(k)w(k) 

♦  tk|((A(k)  ♦  •(k)0(k)C(k))c(k)(A(k)  ♦  B(k)0(k)C(k))^ 

♦  «k)a(li)»(k)0*(k)/(k)  ♦  ▼{k)-E(k)j  if’^{kvi)j  (3 


In  tb«  trttfifttit  belov,  all  the  vaxlables  at  tine  k  are  denoted  without  *(k}, 
vbKreaa  the  TaLriablea  at  tine  (k^l)  are  denoted  with  *(k‘^l). 

In  (35),  X{k*l)  ia  the  nan  Matrix  of  "coatate  warlablca".  Applying  the 
dleerarte  Matrix  mIdImum  principle  and  the  gradient  Matrix  fonnilae  of  reference 


(8),  the  neceeaary  eonditlona  are 

0  •  -  («  ♦  B^P(kn)B)“^B^P(k^l)Arc^(C£C^  ♦  (36) 

P(k)  •  Q  ♦  C^C^BOC  ♦  (A  ♦  BOC)^P(k^l)(A  ♦  BCiC)  (37) 

Kkn)  •  (A  ♦  BOc)r{k)(A  ♦  aoc)^  ♦  BtswcV  ♦  v  (38) 

P(B)  •  8  (39) 

£(0)  •  £q  (*»0) 


atiera.  ae  la  the  eoatlanoua  tlae  ease,  P(k)  ■  2K(k)  ■  P^(k).  Bote  that  in  the 
dlaerMtM-tUae  eaae,  it  hae  beea  poealble  to  obtain  G  explicitly  in  (36).  (It 
la  aaereil  tlaat  X(k)  le  poeltlee  deflalte,  C  hae  rank  t,  ao  that  C£(k)C^,  la 
poeitlee  deflalte.  A  attmelaBt  eoadltlon  for  I(k)  to  be  poeitlwe  definite  for 
k  >  0  le  that  X(0)  Is  positive  definite,  althov«b  this  ia  not  oeeeasary. ) 

HoMtlOMS  (96)  to  (40)  deflae  a  poMllpeer  Matrix  teo  point  boundary~Talue 
prOhlMM.  It  eea  be  verified  that  these  eoodltloaa  are  e^valent  to  those  ob¬ 
tained  U  refereaee  (T)  eslnc  dyaoMle  progrenlac. 

Xaflalte  Doretioe  Case 

As  la  the  eoatlaasas  tlae  ease,  if  A,  B,  C.  V.  W,  and  0  are  constant  and 
(A  ♦  ne)  Is  Stahls,  it  can  be  shovn  that  the  state  covarlaace  Matrix  ^  > 

0  as  h  •  and  saUsfles  (36)  elth  •  E^.  furthersore,  it  is  assuMsd  that 
I  is  positive  definite  for  the  V,  V,  ond  0  and  the  dywsMlca  under  eonaideration. 
this  tsoold  indeed  be  true  if  (f  ♦  is  positive  definite.  As  in  ttM 

eoMtisnoiis  ease,  the  Modified  perforMsaee  function  ia 

3  -  llM  J  I  I  Ctx’(k)Qx(k)  ♦  u'^(k)Ru(k)J  (41) 

■  k-0  ^ 
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or 


J  «  ^  Tr  ♦  c'^g'^RGC)^  J  ♦  ^  Tr(G^mV)  (»*2) 

For  this  otAtie  ofrtlMiMtion  problem  baviog  tbe  constraint  (38)  with 

■  2,  doflns  tbo  Lasranglan  as 

Kt.O.t)  •  I  Tr  [(Q  ♦  C^O^BOOt  j  ♦  |  Trlo'^ROWj 

♦  *‘[{<A  ♦  B0C)t{A  ♦  BGC)^  ♦  BCWgV  ♦  V  -  (1*3) 

viMHro  t  is  ths  B  X  n  Mtrix  of  Lagrange  Bultlpllers.  The  necessary  conditions 

9M  9B 

are  obtalsed  by  e^iuatlnc  — ,  fr,  and  —  to  tero: 

ar  aR 

0  •  .(R  ♦  b*pb)'H^paIc^(cIc^  ♦  w)“^  (UU) 

P  -  Q  ♦  cVbOC  ♦  (a  ♦  BOC)’p(A  ♦  BOC)  (>*5) 

t  -  (A  ♦  BOC)Z(A  ♦  BCC)’  ♦  V  ♦  BOUcV  (1*6) 

etaare,  aa  la  the  eostlBMoua  ease,  P  •  2*  •  p’.  This  Is  a  set  of  iK>''*near 
■atrlx  algahrair  eqtiaatlotta.  Slaee  R  >  0,  C  >  0  and  rank  (C)  ■  t,  the  Inverses 
ia  ikk)  axlat. 

A  luMrrlcal  Algorltha 

nwieHarlm  flret  the  flaite  tanslaal  tine  cane,  let  0^(k)  and  0^(k)  be 
t«s»  feedbaek  pdiie.  sveb  that 

t^(feal)  •  (A  ♦  10^C)t^(k)(A  ♦  lO^C)^  •  BoWV  ♦  V.  1  •  0,  1  (b?) 


P®(k)  -  Q  ♦  C^O®*»0°C  ♦  (A  ♦  I0®C)V(kel)(A  ♦  B0°( 


C) 


(W) 


After  a  leaftbqr  Maaiimlatloa  an  ovtlinod  ia  Appendix  B,  it  can  be  prored 
J(0®)  -  J(O^)  •  I  ♦  W)|(0p  ♦  f)*(R  ♦  BV(kel)B)((/^  ♦  g) 

-  (0^  ♦  •)’<«  ♦  BV{lrH)(0^  ♦  g)|j  (b9 


Ik 


f  •  (R  ♦  BV(k-*'l)B)'HV(lt+l)Ar^c'^(Ci:^c'^  +  W)"^ 


(50) 


In  vrltla*  equations  (47)  through  (50), ‘(k)  has  been  dropped  in  most  places  for 
eooeenlenee,  except  lAum  required  for  clarity.  Letting 

-  -g  (51) 

in  (49),  we  have 

J(G°)  -  J(G^)  >  0  (52) 

Thus,  a  Blnljalxlog  algoritte  la 

(a)  choose  initial  G^(k) 

(b)  obtain  P®(k),  k  ■  froa  the  linear  equation  (48) 

(c)  obtain  r^(k)  by  solving  the  nonlinear  difference  equation  obtained 

by  eubetitutlon  of  (51)  In  (4?) 

(d)  obtain  0^(k)  using  P^(kel)  and  C^(k)  detemlned  In  the  above  steps 

(e)  go  to  (b),  with  the  superscript  raised  by  unity 

Thus  a  sueoeesive  redueti<m  in  J  is  obtained,  for  the  infinite  terminal 
tiae  ease,  after  a  slBllar  ■enlpulation  (given  In  the  Appendix  B),  it  can  be 
shown  that 

J(0®)  -  J(O^)  -  jTrfctc^  ♦  ¥)|«P  ♦  g)^(R  ♦  bVb)(C^  ♦  g) 

-  (0^  ♦  g)^{R  ♦  bVb)(0^  ♦  g)Jj  (53) 

wbere  end  P^  satisfy  the  static  versions  of  (47)  and  (46),  and 

g  •  (R  ♦  BVB)'‘Vp®Al^^(cr^C^  ♦  W)*^  (54) 

nms  J(O^)  <  J(O^)  with  0^  ■  the  resulting  sequence  of  0^'s 

iaqproves  the  value  of  J  successively  if  each  (A  ♦  BOC)  is  stable,  sad  each 

t  >  0. 

flms,  a  a»erloal  al^riths  similar  to  that  for  the  finite  terminal  time 
ease  Is  (Stained. 
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Th*  piraetiealljr  i^portaat  problMi  of  obtoi&lDc  aja  optiaal  output  feedback 
control  lew  for  linear  ^etene  subjected  to  both  forcing  ezMl  neaaureaMnt  noise 
was  emuildered.  leeessary  conditions  for  optinelity  were  obtained  for  continue 
oue-tljse  sad  dlseret#>tine  syst«u,  using  the  aatrix  ninlaua  principle.  Both 
finite  sad  Infinite  teminsl  tine  esses  were  considered  for  each  problen.  Al> 
goarlthM  were  derlwed  for  obtaining  sequences  of  feedback  gains  idxich  guarantee 
a  araotonle  iagprowenent  of  the  perfomaace  function.  The  aethod  developed 
provides  a  powerful  and  practically  feasible  design  technique  in  which  the  a- 
priori  knowledge  of  the  noise  statistics  is  used  to  advantage. 

Although  the  necessary  conditions  for  a  ninlnun  were  obtained,  the  question 
of  eadsteaee  Is  still  unanswered,  and  needs  further  attention.  Also,  the  con¬ 
vergence  properties  of  the  aequencee  of  feedback  galas  generated  deserve  nore 
attention. 

For  the  infinite  temiaal  tine  case,  it  was  assvsMd  that  I  >  0.  The  case 
Z  •  0  will  be  a  slagde  estensloD  of  the  work  of  NcLane  (reference  (6));  however, 
the  ease  Z  >  0  needs  further  investigation.  Also,  the  teclmique  developed  in 

en 

this  paper  can  be  easily  nodifled  for  the  design  of  optinal  dynasde  eoaqpensators , 
ky  foUowiiui  a  procedure  sinilar  to  that  by  Levine  et.  al.  (reference  (k)). 


APPZIDIX  A 


ALGOftZTlM  Foil  SUCCESSIVE  REDUCTIOI  OF  TSE 
FBtFORNABCE  FUBCTIOI:  COVTIKUOUS  TIME  CASE 


Sine* 


Ajtird®.  r*)p®|  .  Tr  j^t®.  r^jj  p®  .  TrCt®. 


(A-l) 


Aft«r  •lA^lflcfttlovi  uslnf  equation  (21)  and  (22)  and  Intesratlng  both  sides 
batvsan  0  and 

J(0®)  -  J(O^)  ■  Tr  (B(G°-  G^)cr^P°J  dt 


♦  ^Tr  (r^(C^C°^RG°C  -  cV^RG^O)  dt 

♦  I  Tr  ^  '  B(0®HO”  -  cH«^^)bV  dt  (A-2) 

til#  trac#  Idantitl##  in  r#f#r#iice  (8),  It  can  b#  proved  that 
•quatioD#  (A-a)  and  (23)  ar#  •quivalent. 


Infialt#  Teralnal  Tlsw  Case 

la  tbls  eaa#,  tb#  •qaatloos  (13)  and  (IW)  take  on  their  steady-stat#  forms 
vlth  t  replacing  C.  tow. 

TV(0‘P®J  ♦  Tr{{t®-  r^)Oj  -  0  (A-3) 


(A  ♦  10®C)l®  ♦  I®(A  ♦  BO®C)*  ♦ 

-  ({A  ♦  BO^Ol^  ♦  t^(A  ♦  BO^O*  ♦  BoSlO^Vl  »  0  (A-k) 


tad 

.(A  ♦  BO®C)V  -  F°(A  ♦  BO°C)  -  (Q  ♦  C*0®^R0°C)  •  0  (A-5) 
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K^Iaeliic  the  first  SAd  second  null  aatrices  in  (A>3)  by  left-hand  sides 
of  (Ar-h)  end  (A*}),  respectively,  after  simplification, 

J(0®)  -  J(O^)  •  Tr{B(G®-  G^)ct^®l  ♦  |  Tr[ r^(C^G°^RG°C  -  C^O^^RG^C)] 

♦  ^Tr  -  gHc^'^)B^1P°)  (A-6) 

Similar  to  the  finite  terminal  time  case  (A-6)  can  be  shown  to  be  equivalent  to 
(23),  vith  intecral  signs  removed,  and  2^  replaced  by 
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AFPBa>IX  B 

« 

ALOQRITSI  rOR  8UCCK8IVE  REDUCTIOM  OF  THE  PERFORMAECE 
FUICTIOf ;  DISCRETE-TIME  CASE 


Dm  «qprMsloB 

Trl{f®(kn)  -  !:^(k>l))P°(k>l)J  -  Tr((E°(k)  -  Z:^(k))P°(k)] 

CM  hm  showB  to  bo  o^|u*l  to  the  folloviois  exprootion,  after  aubatitutlon  for 
Cl®(kol)  -  r^(koi))  fro«  (k?)  and  for  P^{k)  froo  (k6),  reapoctlveiy ,  In  its 
flrot  oad  aocood  torM,  and  after  com  aaolpulatloo: 

-  O^IG^^)BV(kn)J  -  2  Tr(Ar^C^(C°-  C^)Vp°(kel)) 

-  Trtr^C^(O^Vp°(kn)K^  -  0®*BV(k^i)BC®)C] 

-  TrC(t®  -  r^)(C^0®%0®C  ♦  Q)J 

After  oiMilag  tbo  tvo  eacprooelooc  aboee  froa  k  •  0  to  M  -  1,  and  after 
COM  MatBttXatlott,  tlw  followinc  oq^ticm  le  obtained 

J<0®)  -  J<0^)  •  I  V  trUB0®W®^B*  -  BoWV)P°(k^l)J 
^  kiK) 


-  E  TflAT^C^tO®  -  0^)Vp®(lrel)) 
k-0 


♦  i  I  Trtr^C^tO^^BVdt^l)  B0°  -  O^Vp®(kel)  BO^)C] 

*  kM) 

♦  I  *£^  Trt£^C^(0®l«®  -  0^’lW^)Cl 

*  k-0 


♦  I  E  Tf((0°*IW®  -  C^'^RO^)W) 

*  k-0 
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UsinK  the  tr*ee  Identitie*  in  reference  (8),  it  can  be  shown  that  (B-l) 

and  (^9)  are  equivalent. 

Infinite  Tenainal  Tiae  Case 

In  this  ease  equations  (37)  and  (38)  becoa^e  equations  (i*5)  and  (^*6), 
respectively.  low, 

Tr((f^-r^)P®l  -  Trf (!^-r^)P°)  •  0 

Substitutioa  for  (t^-  E^)  in  the  first  tens  using  equation  (1*6),  and  for  F  in 
the  secOTd  tera,  using  equation  (1»5),  and  proceeding  exactly  as  for  the  infinite 
teralnal  tiae  ease,  equation  (53)  can  be  obtained. 
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